Searching PAJ 



Page 1 of 2 



PATENT ABSTRACTS OF JAPAN 

(1 1 )Publication number : 07-067028 
(43)Date of publication of application : 10.03.1995 



(51)Int.CI. H04N 5/235 



(21) Application number : 05-213210 (71)Applicant : SONY CORP 

(22) Date of filing : 27.08.1 993 (72)Inventor : KONDO TETSU JIRO 

HORISHI MASARU 
FUJIMORI YASUHIRO 
NISHIKATA TAKEHARU 



(54) DEVICE AND METHOD FOR RESTORING IMAGE 

(57)Abstract: 

PURPOSE: To restore a degraded image by classifying 
blocked data into plural classes, obtaining class codes 
corresponding to the classification and outputting an image 
without degradation corresponding to this class code. 
CONSTITUTION: A blocking circuit 2 divides an input image, 
which is supplied through an input terminal 1 and starts 
degradation, into plural blocks. A class code generating 
circuit 3 classifies the blocked data into the plural classes 
and outputs the class codes corresponding to the 
classification. A ROM table 4 outputs the image without 
degradation corresponding to the input image corresponding 
to the class code. In this case, the result of previously 
learning with the data, for which the image without 
degradation is divided into blocks, and the data classifying 
the image, for which the image with no degradation divided 
into blocks is degraded with a prescribed degradation 
function, into classes is stored in the ROM table 4 as a 
table corresponding to the class code. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



EFFECT OF THE INVENTION 



[Effect of the Invention] As mentioned above, the resuh of having learned beforehand the image with 
which image quality deteriorated in this invention with the data which carried out the class classification 
for every block, and the block data of an image without degradation is memorized. Since he is trying to 
output the image which divides a degradation image into two or more blocks, classifies the blocked data 
into two or more classes, obtains the class code corresponding to that classification, and does not have 
degradation corresponding to a degradation image according to this class code Even if it is the case 
where the cause of degradation cannot be specified, and the case where an inverse function cannot be 
determined uniformly, restoration of the image which deteriorated is attained. 



[Translation done.] 
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TECHNICAL PROBLEM 



[Problem(s) to be Solved by the Invention] By the way, various kinds of things, such as what is 
depended on a uniform motion of a camera (image photography equipments, such as a video camera) as 
a cause of the above-mentioned degradation, and dotage by the optical system of a camera, can be 
considered. Therefore, when restoring the above-mentioned image, the model function of the above- 
mentioned degradation will also change with causes of each degradation. 

[0004] If the cause of the degradation cannot be specified when it is going to restore degradation since it 
is such, the above-mentioned model function cannot be determined but restoration will become difficuh. 

[0005] Moreover, even if it is able to stand the model function of the cause of degradation, there is a 
case where the inverse function for restoring does not exist, plentifully, and evaluation of the optimal 
model in that case is also difficult. 

[0006] That is, in restoring the image with which image quality deteriorated, it restores by analyzing the 
cause of degradation, determining a degradation model function, and giving the inverse function, but 
restoration becomes difficult when neither the case where the cause of degradation cannot be specified 
as mentioned above, nor an inverse function can be determined uniformly. 

[0007] Then, even if this inventions are the case where the cause of degradation cannot be specified in 
view of what mentioned above, and the case where an inverse function cannot be determined uniformly, 
they aim at offering the image restoration equipment and the approach which can perform restoration of 
the image which deteriorated. 



[Translation done.] 
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MEANS 



[Means for Solving the Problem] This invention is characterized by providing the following. It is a 

blocking (scan conversion of input image is carried out) means to be proposed in order to attain the 
above-mentioned purpose, and to divide into two or more blocks the input image which the image 
restoration equipment of this invention is equipment which restores the image with which the image 
quality of a digital image signal deteriorated, and has produced degradation. A class classification means 
to classify the blocked data into two or more classes, and to output the class code corresponding to the 
classification A non-deteriorated (data by which study optimization was carried out beforehand are 
outputted) image output means to output the image which does not have degradation corresponding to 
an input image according to the above-mentioned class code 

[0009] Here, the above-mentioned non-deteriorated image output means has memorized the result (study 
value) beforehand learned by the data which blocked the image without degradation, and the data which 
gave the class classification to the image which degraded the image without the blocked degradation 
concerned with the predetermined degradation function as a table to which the above-mentioned class 
code was made to correspond. Or the above-mentioned non-deteriorated image output means has 
memorized the result (study value) beforehand learned by the data which blocked the image which 
deteriorated and carried out the class classification, and the data which blocked the image without 
degradation as a table to which the above-mentioned class code was made to correspond. 
[0010] Moreover, the image-restoration approach of this invention comes to have the class classification 
procedure which classifies into two or more classes the data blocked the block chemically-modified 
[ which divides into two or more blocks the input image which has produced degradation ] degree, and 
outputs the class code corresponding to the classification, and the non-deteriorated image output process 
which output the image which does not have degradation corresponding to an input image according to 
the above-mentioned class code. 

[001 1] Here, at the above-mentioned non-deteriorated image output process, the study value 
corresponding to the above-mentioned class code is taken out from two or more study values which 
learned beforehand and were acquired by the data which blocked the image without degradation, and the 
data which gave the class classification to the image which degraded the image without the blocked 
degradation concemed with the predetermined degradation function. Or at the above-mentioned non- 
deteriorated image output process, the study value corresponding to the above-mentioned class code is 
taken out from two or more study values which learned beforehand and were acquired by the data which 
blocked the image which deteriorated and carried out the class classification, and the data which blocked 
the image without degradation. 



[Translation done.] 
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OPERATION 



[Function] According to this invention, the result (restoration table) of having learned beforehand the 
image with which image quality deteriorated, and having obtained it with the data which carried out the 
class classification for every block, and the block data of an image without degradation is saved. That is, 
since the table by study is learned between the image which deteriorated, and the image without 
degradation, it can obtain the image with which degradation was restored by the thing which take out the 
study value corresponding to the class code by class classification of the image which deteriorated from 
this table (the restoration image which does not have degradation instead of the image which 
deteriorated applies (mapping)) and which makes like. Moreover, once it creates a table, since 
restoration of an image is only processing which reads a table, it will become easy to process it. 



[Translation done.] 
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EXAMPLE 



[Example] Hereafter, it explains, referring to a drawing about the example of this invention. 
[0014] As the image restoration equipment of the 1st example of this invention is equipment which 
restores the image with which the image quality of a digital image signal deteriorated and is shown in 
drawing 1 The blocking (scan conversion of input image is carried out) circuit 2 which divides into two 
or more blocks the input image which has produced degradation supplied through an input terminal 1, 
The class code generating circuit 3 which is a class classification means to classify the blocked data into 
two or more classes, and to output the class code corresponding to the classification, It comes to have 
the ROM table 4 which is a non-deteriorated (data by which study optimization was carried out 
beforehand are outputted) image output means to output the image which does not have degradation 
corresponding to an input image according to the above-mentioned class code. 
[0015] The result (a study value) which learned beforehand by the data which gave the class 
classification to the image degraded with the predetermined degradation function which mentions the 
data which blocked the image in which the above-mentioned ROM table 4 does not have degradation 
here, and an image without the blocked degradation concerned later has memorized as a table to which 
the above-mentioned class code made correspond (the 1st example which mentions later). Or the result 
(study value) beforehand learned by the data which blocked the image which deteriorated and carried 
out the class classification, and the data which blocked the image without degradation is memorized as a 
table to which the above-mentioned class code was made to correspond (the 2nd example mentioned 
later). Moreover, the above-mentioned class code generating circuit 3 asks for the multiplier group of 
the formula showing the configuration nearest to an input image, using a high order polynomial which is 
mentioned later as a model type, and performs a class classification according to this multiplier group. 
[0016] Here, in advance of explanation of the configuration of the image restoration equipment of this 
invention example, generation of the study in image restoration and a ROM table is explained. 
[0017] When the table learned beforehand tends to perform image restoration, it is necessary to carry out 
study processing between the image which deteriorated, and a beautiful image without degradation. As 
an approach of obtaining the degradation image used for study, the photographic subject is fixed, for 
example, a camera is moved and it flows, and an image is generated or there is [ the focus of a camera 
can be shifted intentionally, it fades and ] the approach of generating an image. 
[0018] Here, the 1st example of the flow of the processing when learning using a degradation image is 
shown in drawing 2 . 

[0019] In this drawing 2 , an image without degradation is supplied by step S9. At the following step 
SIO, the above-mentioned degradation-less image is blocked and it sends to step SI 3. Moreover, the 
degradation-less image of step SIO is degraded using a known degradation ftinction in step SI 1 . Then, 
clustering which classifies into a class the block data degraded with the above-mentioned degradation 
fimction according to step S12 is performed, and it progresses to step 813. In addition, a class 
classification is a pattern classification using the blocked image data. 

[0020] At step SI 3, it trains between block data without degradation fi-om the above-mentioned step 
SIO, and the block data which was degraded from the above-mentioned step S12, and carried out the 
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class classification (study). 

[0021] At step SI 4, it judges whether study was repeatedly performed to extent to which all class study 
is fully performed to all blocks that constitute an image. If it is judged return and yes by step SIO when 
judged no at this step SI 4, it will progress to step SI 5. 

[0022] Step S15 generates a mapping table with the optimum value leamed for every class. This serves 
as a ROM table. 

[0023] Although the 1st example of the above is generating the degradation image by degrading a 
degradation-less image with a degradation function, like the 2nd example described below, as 
beforehand mentioned above, it can also learn using the degradation image which carried out shifting 
migration of a camera and a focus etc. and was generated, and the flow of processing in this case comes 
to be shown in drawing 3 . 

[0024] In this drawing 3 , an image without degradation of the degradation image generated physically 
beforehand at step S2 is supplied at step SI. At step S3, the above-mentioned degradation image is 
blocked and the above-mentioned degradation-less image is blocked in step S4 in the location 
corresponding to blocking at step S3. 

[0025] At the next step S5 of step S3, clustering which classifies the block data of the above-mentioned 
degradation image into a class is performed. It is the thing of a pattem classification using the image 
data which also blocked this class classification. 

[0026] At step S5 and step S6 which progresses after processing of step S4, it trains by being between 
the blocks of the above-mentioned degradation image and a degradation-less image (study). 
[0027] At step S7, it judges whether study was repeatedly performed to extent to which all class study is 
fiiUy performed to all blocks that constitute an image. When judged no at this step S7, to a degradation 
image, it passes step S3, and returns to tetraethylpyrophosphate S4 to a degradation-less image. If 
judged yes at step S7, it will progress to step S8. 

[0028] Step S8 generates a mapping table with the optimum value leamed for every class. This serves as 
a ROM table. 

[0029] In addition, there is a pattem of some [ how the block of the above-mentioned degradation image 
and the block of a degradation-less corresponding image are leamed ]. For example, it may leam and 
output by the case where it leams and outputs with the block of a degradation image, and the block of a 
degradation-less image, and the pixel. The case where the input image itself is leamed as data 
ftirthermore leamed, and a difference daily dose with a degradation filter may be leamed. 
[0030] Moreover, although live data can also be used for the class in the 1st and 2nd example of the 
above, if the memory space which considered implementability is taken into consideration, it is possible 
[ it ] to use a data compression etc. for a class classification. 

[0031] In a data compression, the adaptive quantization (referred to as ADRC) according to the dynamic 
range of the picture signal mentioned later, DPCM (differential PCM), vector quantization (VQ), etc. 
can be considered. 

[0032] In addition. Above ADRC is a thing which was adapted for the above-mentioned dynamic range 
for which blocked the digital image signal, and asked for the dynamic range specified by two or more 
maximums and minimum values of pixel data which are contained in this block, and it asked in this 
block unit and which assigns and encodes each pixel data within the block concerned with the number of 
bits. Speaking more concretely. Above ADRC asking for the dynamic range (difference of the maximum 
level within a block, and the minimum level) specified by the maximum of two or more pixels and the 
minimum value which are contained in a block predetermined [ in one frame (or field) ]. The adjustable 
quantifying bit number which was adapted for this dynamic range is determined for every above- 
mentioned block, this quantifying bit number - the input data after minimum value removal - coding 
(quantization re—) Namely, while the compressed quantifying bit number divides a dynamic range 
equally, encoding and using each pixel within a block as the code of the nearest level and obtaining a 
coding code It seems that it uses as an addition code any [ the information on a dynamic range, and ] of 
the above-mentioned maximum and the minimum values to be, and these coding code and an addition 
code are obtained. 
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[0033] Furthermore, the following approaches are used for the implementation approach of clustering. 
[0034] Here, as simplest approach, there is the technique of making a class number the bit sequence of 
the study data within a block as it is, for example by class division. However, by this technique, ROM of 
a huge capacity is needed. 

[0035] For this reason, in this example, Above ADRC is used for the class division by the signal pattern, 
and the effective number of classes which saved the property of a signal pattern is reduced. Here, as 
shown in drawing 4 , the prediction to the HARASHIN number pixel A is taken for an example from the 
study signal pixels a, b, and c, and it is the level of the study signal images a, b, and c, respectively xl, 
x2, and x3 It carries out. Moreover, it is re-quantization data of the result of having performed p bit 
ADRC to xl, x2, and the data of x3, respectively ql, q2, and q3 It carries out, and the dynamic range is 
set to DR and the minimum value is set to min. 

[0036] At this time, class number (class code) class of this block is defined by the formula (1). 



[0038] In addition, originally ADRC is the accommodative re-quantizing method developed for [ to 
turn ] high efficiency coding VTR (video tape recorder), and since the local pattern of signal level can 
be efficiently expressed by the short word length, in this example, this is used for code generating of a 
class classification of a signal pattern. 

[0039] More generally it explains. They are xl, xn about study signal pixel level. When it carries out 
and the HARASHIN number pixel level is set to y, they are multipliers wl, wn for every class. 
Linearity presumption type of n tap to depend (2) y==wl xH-w2 x2+ ... +wn xn (2) 
It sets up. In addition, it is wi before study. It is an undetermined coefficient. 

[0040] Study is performed to two or more signal data for every class. When the number of data is m, it is 
a formula (2) (prediction type). It follows and is yk =wl xkl+w2 xk2+... +wn xkn (k= 1, 2, m) (3) 
In M>n, they are wl, wn. Since it does not become settled uniquely, it is [ ] the element of the error 
vector e ek =yk-{wl xkl+w2 xk2+ ... +wn xkn} (4 (k= 1, 2, m)) 

A definition is given and it is a formula (5). It asks for the multiplier made into min. It is a solution 
method by the so-called least square method. 
[0041] 
[Equation 2] 

^2 = S{e,} (5) 

[0042] Here, it is a formula (5). wi About ********** to depend, it is a formula (6). It asks like. 
[0043] 
[Equation 3] 



[0044] The above-mentioned formula (6) It is each wi so that it may be made 0. Since what is necessary 
is just to decide, it is a formula (7). Formula (8) It is [0045] like. 
[Equation 4] 



[0037] 
[Equation 1] 




(1) 




m 




(7) 



[0046] 
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[Equation 5] 



4f 



(8) 



[0047] It is a formxila (9), when it carries out and a matrix is used. It becomes like and is [0048]. 
[Equation 61 



fx .V 



X. 
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X 



nl 
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'n2 



• ■ 



2/1 



A 



Ynj 



(9) 



[0049] sweeping out - general matrix solution methods, such as law, - using - wi ********- if it 
solves, a prediction coefficient can be found and this prediction coefficient is stored as data (ROM table) 
of the address class of ROM in this example. 

[0050] Next, the above-mentioned degradation function is explained below. First, the image observation 
model which took the noise into consideration is considered. The image observation model which took 
this noise into consideration can be expressed like a formula (10). 

g=Af+n (10) In addition, in this formula (10), in f, observation data and A express an observation 
process and, as for n, a subject-copy image and g express the additivity noise. 
[0051] Here, it is the space where the target subject-copy image f belongs HI It is the space where it 
carries out and the observation data g belong H2 Noise n is H2 when it carries out. It is origin and 
Observation A is HI mathematically. H2 It can be regarded as an operator, f and g may be expressed as 
an array which may be expressed as a function defined on the continuation field, and was defined on the 
dispersed point. In the case of the latter, it is the same as a vector. According to these expression 
approaches, the model of a formula (10) is called as follows. Namely, HI When both H2 is functional 
space, it is called a continuation-continuous model, and in both the cases of vector space, it is called 
discrete - dispersion model. Moreover, HI It is H2 in functional space. The case where it is vector space 
is called continuation-dispersion model. 

[0052] In the case of the above-mentioned continuation-continuous model, the above-mentioned formula 
(10) is expressed like a formula (11). 
[0053] 
[Equation 7] 

gU,y) = tirJ'Uy.-^'. v')/('''.v')'i^ V +«U..v) (1 1) 

[0054] In addition, in this formula (1 1), f (x y) is [ a degradation image and n (x y) of a subject-copy 
image and g (x y) ] noises, and h (x, y, x', y*) is called a degradation fimction. for example, f(x y) =delta 
(x-alpha, y-beta) ~ the degradation image to the point light source is h (x y, alpha, beta). Therefore, a 
degradation fimction may be called a point spread function (PSF:point spread function) to coincidence. 
[0055] When the image with which the point light source deteriorated does not call at the location of the 
point except for a parallel displacement here, a point spread function turns into a fimction of h (x-x\ y- 
y') and difference type, and the above-mentioned formula (1 1) is a convolution integral (convolution) 
like a formula (12). It is expressed. 
[0056] 
[Equation 8] 

gUy) = r.rJ^(-^ --^'^y- /)/Cr'. v')^'./v' + «Uy) (12) 

[0057] such degradation - migration unchangeability, location unchangeability, or space ~ it is said that 
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it is eternal. A formula (11) may be convertible for the format of a formula (12) with a change of 
variables. 

[0058] Moreover, degradation may be separable in the direction of a x axis, and the direction of the y- 
axis. Speaking mathematically, the degradation function's supporting that it can decompose into the 
form of a product like a formula (13). 

h(x, y, x', y') =hl h(x x') 2 (y, y') (13) This is called isolated nuclei. Bure of the photograph by uniform 
motion of a camera - isolated nuclei and migration - it is the example of eternal degradation. 
[0059] Moreover, discrete - dispersion model of the above-mentioned formula (10) is as follows. For 
example, N dimension vector made from the value in the sample point of N individual of the subject- 
copy image f (x y) is set to f g, n, then the above-mentioned formula (1 1) are the degradation image g (x 
y) and Noise n (x y), and N dimension vector made in approximation similarly, respectively g=Af+n It 
can express (14). Here, A in a formula (14) is the Nth square matrix showing degradation. 
[0060] Moreover, a sample point is Nl xN2. When given in the lattice point (xm and yn) of an 
individual (N=N1 xN2), the sampled value f (xm and yn) of f (x y) can be put in order as it is, and can 
be expressed with the form of a matrix. This is set to [f]. It is similarly referred to as [g] and [n] to the 
degradation image g (x y) and Noise n (x y). Furthermore, suppose that they are Degradation A isolated 
nuclei like a formula (13). At this time, the formula (1 1) of a continuation-continuous model is discrete - 
dispersion model in approximation, [g] =A1 [f] A2+ [n] It can be expressed as (15). Here, it is A 1 and 
A2. It is Nl, respectively. A degree and N2 It is degree square matrix. 

[0061] furthermore, the degradation A - migration - case it is eternal - a formula (12) - receiving — 
Al of a formula (15), and A2 Tape RITTSU (Toeplitz) If approximated in procession, it is convertible 
for a diagonal matrix with discrete Fourier transform (DFT). This is equivalent to changing the 
convolution integral of a formula (12) into the form of a product with the Fourier transform. 
[0062] Next, after degradation by the continuation-continuous model, a continuation-dispersion model 
performs sampling and changes it into Vector g or a matrix format [g]. Therefore, in the part of the first 
half, it is a formula (11), The expression of a formula (15) can be used. Or these two processes can be 
summarized and it can also express like a formula (16). 
[0063] 
[Equation 9] 



[0064] this formula (16) - gij and nij - respectively - the [ of [g] and [n] ] - it is ij component. 
[0065] In addition, as shown in the above-mentioned formula (1 1) - the formula (19), there are many 
approaches of expressing an image degradation model concretely. 

[0066] It retums to drawing 1 and the example of a configuration of the image restoration equipment of 
the 1st example which reads data from the ROM table which learned as mentioned above and was 
obtained, and performs image restoration is explained. The image restoration equipment of drawing 1 
shows the configuration in the case of outputting the pixel which restored degradation to the input 
degradation image. 

[0067] In this drawing 1 , the digital image signal which was inputted into the input terminal 1 and 

which deteriorated is sent to the blocking circuit 2. The supplied digital image signal is divided into two 

or more blocks which consist of two or more pixel data in the blocking circuit 2 concerned. 

[0068] The block data from the above-mentioned blocking circuit 2 is sent to the class code generating 

circuit 3. In the class code generating circuit 3 concerned, it asks for a class with reference to the pattern 

of two-dimensional level distribution of the supplied block, and the class code is generated. 

[0069] This class code is sent to the ROM table 4 as address data. The study value (namely, pixel data 

without degradation) mentioned above is stored in the ROM table 4 concemed, and the study value is 

outputted corresponding to the above-mentioned class code (address). The study value, i.e., pixel data 

without degradation, read from the ROM table 4 concemed is outputted from an output terminal 5. 

[0070] Next, the configuration in the case of saving the study value of a block unit is shown in the ROM 
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table 9 as the 2nd example of this invention at drawing 5 , 

[0071] In this drawing 5 , it is the same configuration as the input terminal 1 of drawin g 1 to the class 
code generating circuit 3 up to the class code generating circuit 8 from an input terminal 6. Here, the 
ROM table 9 of drawing 5 outputs the data of a block unit to the ROM table 4 of drawing 1 outputting 
the data which are not a block unit. 

[0072] The data of the block unit without degradation outputted from the ROM table 9 concemed are 
sent to the block decomposition circuit 10. That is, since the output of the ROM table 9 is a block unit, 
by the block decomposition circuit 10, block decomposition is performed and the disassembled data 
turns into output data of a degradation-less image from an output terminal 1 1 . The non-deteriorated 
image output means consists of this 2nd example in the ROM table 9 and the block decomposition 
circuit 10. 

[0073] Next, as the 3rd example of this invention, about the difference of the output of the restoration 
filter which restores degradation with the inverse function of a degradation function, and the output of 
the same blocking circuit 13 as drawin g 1 and drawin g 5 , when it learns previously, the example of an 
about is shown in drawing 6 . 

[0074] That is, in the class code generating circuit 16 of the image restoration equipment of the 3rd 
example shown in this drawing 6 , it is made to perform class classification processing to the difference 
of the input image blocked through the blocking circuit 13, and the output which filtered that block 
image through the restoration filter 14. 

[0075] In this drawing 6 , a digital image signal is supplied to an input terminal 12 like the input 
terminals 1 and 6 of drawin g 1 and draw ing 5 , and blocking does in the same, same blocking circuit 13 
as drawing 1 and drawin g 5 . 

[0076] The output of the blocking circuit 13 concemed is sent to the restoration filter 14 which restores 
degradation of an image with the inverse function of a degradation function. The output with which 
filtering was made with the filter 14 concerned is sent to an adder 15 as a subtraction signal. The 
difference of the block data which the output from the above-mentioned blocking circuit 13 was 
supplied to the adder 15 concemed also as an addition signal at this time, therefore was filtered in the 
adder 15 concemed, and the block data which is not filtered will be called for. 

[0077] The output of this adder 15 is sent to the class code generating circuit 16. It asks for a class with 
reference to the pattern of two-dimensional level distribution of the output block of the above-mentioned 
adder 15, and the class code is generated also in the class code generating circuit 16 concerned. 
[0078] This class code is sent to the ROM table 1 7 as address data. The study value which learned by 
the data of the difference of pixel data without degradation and the data which filtered pixel data without 
the degradation concemed with the degradation filter is stored in the ROM table 17 concerned, therefore 
a study value comes to be outputted corresponding to the above-mentioned class code (address). 
[0079] The study value outputted from this ROM table 17 is sent as an addition signal as well as the 
adder 1 8 with which the output of the above-mentioned filter 14 is supplied as an addition signal. 
Thereby, from the adder 18 concemed, pixel data without degradation are outputted and this is outputted 
by the output terminal 19. Therefore, the non-deteriorated image output means is constituted from this 
3rd example equipment by the above-mentioned ROM table 17 and the adder 18 with which the output 
of a filter 14 is supplied. 

[0080] In addition, although it is made to give a class classification to said difference with the image 
restoration equipment of the 3rd example, it is also possible to perform a class classification, for 
example to the output of the filter 14 of afi input image. 

[0081] As mentioned above, it sets to the image restoration of this invention example. Carry out scan 
conversion (blocking) of the input image with which image quality deteriorated, and class generation is 
performed from this block image data. By outputting the data (data by which study optimization was 
carried out) which act on image restoration from the table beforehand leamed according to the class 
(namely, signal pattern), and generating a restoration image from the data Even if it is the case where the 
image with which degradation was restored can be obtained, therefore the cause of degradation cannot 
be specified, and the case where an inverse function cannot be determined uniformly, restoration of the 
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image which deteriorated is attained. 

[0082] Namely, although it restores by analyzing the cause of degradation, usually determining a 
degradation model function, and giving an inverse function when it is going to restore the image with 
which image quality deteriorated A sake [ when the ability to determine uniformly neither the case 
where the cause of degradation cannot be specified, nor an inverse function, in this invention example ] 
Instead of using the inverse function of a degradation model function, he is trying to restore an image 
using the restoration table beforehand obtained by study by the thing which apply the restoration image 
which does not have degradation instead of the image which deteriorated (mapping) and to do. 
[0083] Thus, in order to restore an image by mapping on the study table using an image without 
degradation, once it is necessary to ask neither for the decision of a complicated degradation function, 
nor its inverse function upwards and creates a table in this invention example, since it is only reading the 
table of image restoration, processing is easy to process. Moreover, the inverse function of a degradation 
function cannot be defined, but if it is the former, when it cannot restore, a restoration table can be 
created by study. 

[0084] Therefore, according to the image restoration equipment of this example, the image with which 
image quality degradation to the image picturized, for example with the television camera was restored 
can be obtained. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2,**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block circuit diagram showing the outline configuration of the image restoration 
equipment of the 1st example of this invention. 

[Drawing 2] It is the flow chart of the 1st example about the study in this invention example. 
[Drawing 3] It is the flow chart of the 2nd example about the study in this invention example. 
[Drawing 4] It is drawing for explaining prediction of a study signal image to the HARASHIN number 
image. 

[Drawing 5] It is the block circuit diagram showing the outline configuration of the image restoration 
equipment of the 2nd example. 

[Drawing 6] It is the block circuit diagram showing the outiine configuration of the image restoration 

equipment of the 3rd example. 

[Description of Notations] 

2, 7, 13 .... Blocking circuit 

3, 8, 16 .... Class code generating circuit 

4, 9, 17 ....ROM table 

10 Block decomposition circuit 

14 Filter 
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DRAWINGS 
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CLAIMS 



[Claim(s)] 

[Claim 1] The image-restoration equipment carry out [ coming to have in a blocking means divide into 
two or more blocks the input image which has produced degradation, a class classification means 
classify the blocked data into two or more classes, and output the class code corresponding to the 
classification, and a non-deteriorated image output means output the image which does not have 
degradation corresponding to an input image according to the above-mentioned class code, and ] as the 
description. 

[Claim 2] The above-mentioned non-deteriorated image output means is image-restoration equipment 
according to claim 1 characterized by to have made the above-mentioned class code correspond and to 
have memorized the result learned beforehand by the data which blocked the image without degradation, 
and the data which gave the class classification to the image which degraded the image without the 
blocked degradation concemed with the predetermined degradation function. 

[Claim 3] The above-mentioned non-deteriorated image output means is image restoration equipment 
according to claim 1 characterized by having made the above-mentioned class code correspond and 
having memorized the resuh leamed beforehand by the data which blocked the image which deteriorated 
and carried out the class classification, and the data which blocked the image without degradation. 
[Claim 4] The image-restoration approach which carries out [ coming to have the class classification 
procedure which classifies into two or more classes the data blocked the block chemically-modified / 
which divides into two or more blocks the input image which has produced degradation / degree, and 
outputs the class code corresponding to the classification, and the non-deteriorated image output process 
which output the image which does not have degradation corresponding to an input image according to 
the above-mentioned class code, and ] as the description. 

[Claim 5] The image-restoration approach according to claim 4 characterized by to take out the study 
value corresponding to the above-mentioned class code from two or more study values which leamed 
beforehand and acquired by the data which blocked the image without degradation at the above- 
mentioned non-deteriorated image output process, and the data which gave the class classification to the 
image which degraded the image without the blocked degradation concemed with the predetermined 
degradation function. 

[Claim 6] The image restoration approach according to claim 4 characterized by taking out the study 
value corresponding to the above-mentioned class code from two or more study values which leamed 
beforehand and were acquired by the data which blocked the image which deteriorated at the above- 
mentioned non-deteriorated image output process, and carried out the class classification, and the data 
which blocked the image without degradation. 
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[Document to be Amended] Specification 

[Item(s) to be Amended] Claim 

[Method of Amendment] Modification 

[Proposed Amendment] 

[Claim(s)] 

[Claim 1] A blocking means to divide into two or more blocks the input image which has produced 
degradation, 

A class classification means to classify the blocked data into two or more classes, and to output the class 
code corresponding to the classification, 

Image restoration equipment characterized by coming to have a non-deteriorated image output means to 
output the image which does not have degradation corresponding to an input image according to the 
above-mentioned class code. 

[Claim 2] The above-mentioned non-deteriorated image output means is image-restoration equipment 
according to claim 1 characterized by to have made the above-mentioned class code correspond and to 
have memorized the result learned beforehand by the data which blocked the image without degradation, 
and the data which gave the class classification to the image which degraded the image without the 
blocked degradation concemed with the predetermined degradation function. 
[Claim 3] The above-mentioned non-deteriorated image output means is image restoration equipment 
according to claim 1 characterized by having made the above-mentioned class code correspond and 
having memorized the result learned beforehand by the data which blocked the image which deteriorated 
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and carried out the class classification, and the data which blocked the image without degradation. 
[Claim 4] It has further the restoration section which restores degradation of image data with the inverse 
function of a degradation function, 

difference with the data with which blocking of the above-mentioned class classification means was 
carried out [ above-mentioned ] with the output data from the above-mentioned restoration section ~ the 
image restoration equipment according to claim 1 characterized by classifying into two or more classes 
the data by which blocking was carried out [ above-mentioned ] based on a value. 
[Claim 5] A block chemically-modified [ which divides into two or more blocks the input image which 
has produced degradation ] degree. 

The class classification procedure which classifies the blocked data into two or more classes, and 
outputs the class code corresponding to the classification, 

The image restoration approach characterized by coming to have the non-deteriorated image output 
process which outputs the image which does not have degradation corresponding to an input image 
according to the above-mentioned class code. 

[Claim 6] The image-restoration approach according to claim 5 characterized by to take out the study 
value corresponding to the above-mentioned class code from two or more study values which learned 
beforehand and acquired by the data which blocked the image without degradation at the above- 
mentioned non-deteriorated image output process, and the data which gave the class classification to the 
image which degraded the image without the blocked degradation concerned with the predetermined 
degradation function. 

[Claim 7] The image restoration approach according to claim 5 characterized by taking out the study 
value corresponding to the above-mentioned class code from two or more study values which learned 
beforehand and were acquired by the data which blocked the image which deteriorated at the above- 
mentioned non-deteriorated image output process, and carried out the class classification, and the data 
which blocked the image without degradation. 

[Claim 8] It has further the restoration process which restores degradation of image data with the inverse 
function of a degradation function, 

difference with the data with which blocking of the above-mentioned class classification procedure was 
carried out [ above-mentioned ] with the output data from the above-mentioned restoration process — the 
image restoration approach according to claim 5 characterized by classifying into two or more classes 
the data by which blocking was carried out [ above-mentioned ] based on a value. 



[Translation done.] 
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ai««rHi tu. «a!ix-i'g©«-rs^iBjsH2 t-r 

3««nttH! ©TC-PftO, fijM»*AttjB:^Wlc* 
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[0 0 5 2] ±iHS(io)a, ±fBa«-a«*7*ji'©« 
^. ^(iDffljcaic^ssn-s. 

[0 0 53] 
[Sfc7] 



[0 0 5 4] t©it(ll)(C*HT, f (x, y) 

fiUlIfg!. g (x. y) n (x, y) \m 

^Ta&D, h (x. y, x', y') tt^efl^HSctHftf 
n«t>®T**. tflAtf. f (x. y) =6 (x-a, 
y-B) «R«^3tiIlC3*-r««fl:lB«*th (x. y. 



a, 3) Tab 5. Lfc*ttjT, 5&^fc;BlftBll^C, An 
5**0^18: (PSF rpolnt spread function) tljtftl 

[0 0 5 5] Jic:-??, ;jajfcifi©»fl:U&«*^Vff»»* 

50 l»V>T. ^©jSWttlllCiSifttrit*. ji5r>5««?lM» 
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Kh (x-x' , y-y' ) 
n) Tf«*n«. 



(6) 

tmmommzifi*). ±13 * [0 0 5 6] 

a*-ji*«li^(convoIutlo [&81 



7-67028 



10 



gU3') = ^.^-A(^-•«^v•-y')/(.r^v^)^i*V + «U.y) (12) 



[0 0 5 7] ;i<Oi5fe5)5fl:Sr»ii^«. fiff^FS. ^ i 
^«^ij|lcJ:oT, S;(ll) 

si5:(i2)©}j^sic^Si-cirsii-&'b*-5. 5s 

h (x, y, x' , y' ) =hi (x, 

[0 0 5 9] ±iBit:(io)©i8»-««^x;p«. 
*roi3tc*«.. mmmf (x, y) onihcd 

■5. H^lC^&ftHlftg (x. y) RUmmn (x, y) 
g=Af+n (14) 

ta-rctjjtT*^. CUT. iC(i4)»c*tt5Att. ^e* 

Cg] =Ai CfD A2 + CnD 

t&sxti>. Ai . Ai it. -lE-n-enNi j*: 
[0 0 6 1] ^e^isAAt^iii^a:©^^. s;(i2) 

.IC5t*tT. iC(15)0Ai , Aj ttx-T'Uu/^y (Toepli 

tz) nn-vi&mttiii. Hitt7-ux^» (dft) 



gi J = Ho r„/»(j:.y)/(-«^' y)dxdy + 



! [0 0 5 8] Sfc. ^)&fl:*x||fc&|S]tyffc5|6]C^^«nf 

x' ) h« (y. y' ) (13) 

[0 0 6 0] Sfc, g*;S*tNi XNj (N = Ni X 
N2 ) Ofe^jS (x. . y. ) T4AenTVi5S-&. 

f (x, y) ©gi*iif (x. . y. ) s-erosaafe'^ 

«ibH«g (x. y) RUSi'Sn (x. y) lC*fU 
Tfe. HttlC Cg] . Cn] tf*. a&JC. ^UftAtt 

SC(13)C0±5tC^)-|ilS»C?ir.TVi*t-r.5. rcotg. 

aiig-aistxJi'®S(ii) ttj£«tMici8St-«»tx;i' 

(15) 

t4- [0 0 6 2] -Aiz. mm-tmm^ty'Mt. mm-mm=E 

ffmm^ Cg] (XiESlU«:t)0-C»S. Lfc*toT. SJ 
¥©SS5J-lC«a(ll), S5;(15)coSSSffi5;ii:*t-?# 
•5. iSt'ni. ;i0 2 0®jae**i:*T. ^(16)® J: 5 

[0 0 6 3] 

(16) 



[0 0 6 4] 2:®5S(16)T, gu&U^nutt, -ttl^^n 
Cg] RtX CnJ ©SBi 

[0 0 6 5] ife*!. ±ie®it:(ii)~i!:(i9)ic^bfcj;5 

[0 0 6 6] HUCStjT. ±SiE®<t3CtT^SUT 
#:tROMx-:/;i/!5>ex-^'SK*!t*bTia«i«5c* 

■r«. 0i©B««7csatt. KUi^itmmzMvr'^ 

[0 0 6 7] COHUCfet^T. X*«^ltCA*Sn 
[0 0 6 8] ±ia:/D^;i^{t:lHl8S2i4ieffl:/n^/4^7*- 



[0 0 6 9] Z.Ot;y7.:2-Y\t, 7 YVT.v'S' 
HjSx-^') *i|&*^$nT*0. ±IB^'7:^3-K (7 

[0 0 7 0] ^It, *?«K©S2 0SI^«a)i:UT. 05 

ttt. ROMx-:/;v9t:/Dyi?mffi©i^Sfli*«i¥ 

[0 0 7 1] c©05ic::ev>T. Xfsm-&l)^^i;v7. 
3-F»*lHlK8S-Ctt. 01©A*«^1*^S!5'7X 
3-H%4IsIK3*Tt|p|«©«|{S-C$.-5. M 

tlt^OiznV. B5®ROM7^-:/J>9tt:/n>yi/# 

[0 0 7 2] ^iEROMT-:^Jl/9*ie.a:^Sn^^lk 
iSU®:/D.yi?^fi:®7^-^'B. r/n-z^/ij-^niKi o 
itsiens. f&to^. ROMx-:/;i/9fflffiAtt. :/ 
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11 

^mm-v\i, ROMx-:/;P9t:/Pu/i'iWSPigKi 0 

[0 0 7 3] mz. 06tCtt. *%l^®S3©^Jg«ajt 
<offl*tiai, 0 5l^fil®:/n.ye7fl:l5lKi 3«Dffl*t 

CO 0 7 4] fJ^lfc-fe, r:<Oa6(C^-ril3®5l}S««JO 10 
[0 0 7 5] r®0 6(XfcV»T, Ktm^l 2iZ\tM 

*««ii^sn, igi;<iai. 05ti^fli©:rnyi'{bii!B5 

[0 0 7 6] Oy^{i:^^ 13 (DtiiMt. 9;im 20 

M&ns. aR7^ji/3'i4t!7-<;v^"j>i^*t;^$n 
t^^mmn^i 5izu±?&:/ayifim»i 3*>e><o 

[0 0 7 7] CfflinJlCSl 50a*tt. {ryXa-\t9i 
[0 0 7 8] dOi^^Xn-Hf*. 7 YVT.'r-i' th 

XROU7—^)vn\z^f>n^, srromt— 

x-^ w ^' ;i/:J"J tJtx-^ t© 

[0 0 7 9] c®ROM7^-y;n 7A»eai:^snfci^ 

2.. Lfc*JoT. ;i©m3©SIJ|«i!ligH-T?tt. ±bSRO 

Mx-::r;p i 7 1. 7^;v^' i 4©m**ifti&an-5liD 

[0 0 8 0] ts.^. m3®^ifi«s|©il«S7cS®-Ctt. 
iaiSS»IC«L,Ti?7X»fS&J8TJ:3JCbTH5*t. 50 



#BB¥7-6 702 8 

12 

m^\iXi3m^<J>y ^ )Vif l 4©ffiAlwML'T4'7XiJ 
[0 0 8 1] ±.mo^^\z. *58?g^JS«s|©li»«7tlc 

^■fk) U C®:/Dyi/li»T-^'Aie.i'7X4^£&fT 
[0 0 8 2] -tut)*,. iag*i^efbLfcia«!^^7cLJ: 

[0 0 8 3] z.<r>^^\z. *«§9llfi5«-ctt, *)j<b©;ft 
SLn-r^ittb, ffl«t;a:^e{bH»©*s^. ^©ja!!Biia^£ 

[0 0 8 4] Ufe)J«oT, **Sfiei!l©li»«[i^gHIC<t 
[0 0 8 5] 

t *{b®»>ii»©:/D 5/ i^T-^ tT^ di^SUfcits 

«S:l3til-T*lr, ^l5lbH«Sa»07'D-vi7lCi>fiJ 
U. :^nyi/^bSnfcx-:5'£ffi»©i'7X(CiJ-ISUT 
-€-®i^SK:it^£:-r5i'7X3-H*#T. iio^^xn 
- Fici£:i;T5e{bil«!(;:m-r-5^i?{b©liViiii«$ffl* 

m 1 ] *fe^©s 1 a>mi&m0im»i&7tmsofmm 

[0 2 ] ^^fB^^HJEfisUC^tett-S^WlCOt^T®^ 1 ©^ 
[03] *«MIIJSWK*lt«^SlC"3l»T©IS2©a 
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(8) 
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13 

[04] ^wm^mm^^mmm»(D^m^mmt^ 



2, 7, 13 

3, 8. 1 6 

4, 9, 1 7 
10.... 
1 4 • • • • 
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mi} 



m2] 
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[04] 
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[05] 



[B6] 
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